A Graphical Representation for Identifier Structure in Logs

Application console logs are a ubiquitous tool for diagnosing system failures and anomalies. While several techniques exist to interpret logs, describing and assessing log quality remains relatively unexplored. In this paper, we describe an abstract graphical representation of console logs called the identifier graph and a visualization based on this representation. Our representation breaks logs into message types and identifier fields and shows the interrelation between the two. We describe two applications of this visualization. We apply it to Hadoop logs from two different deployments, showing that we capture important properties of Hadoop’s logging as well as relevant differences between the two sites. We also apply our technique to logs from two other systems under development. We show that our representation helps highlight flaws in the underlying application logging.